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ABSTRACT 


Ic  is  shown  that  the  framework  of  [4]  can  be  used  to  give 
a  simplified  proof  of  conditions  given  by  Eaves  and 
Zangwill  [1]  (which  weaken  the  uniform  concavity  require¬ 
ment  on  the  objective  function  used  by  the  author  in  [4]) 
under  which  inactive  constraints  may  be  dropped  after  each 
subproblem  in  cutting-plane  algorithms.  The  convergence 
rate  established  in  [4]  is  improved  and  its  application 
extended. 


A  NOTE  ON  CUTTING-PLANE  METHODS  WITHOUT 
NESTED  CONSTRAINT  SETS 


Donald  M.  Topkis 


The  problem  considered  Is  that  of  maximizing  a  real-valued  continuous  function 
f  over  a  nonempty  closed  convex  subset  S  of  E°  .  One  is  given  a  compact*  convex 
set  T  containing  S  .  The  general  algorithm  to  be  considered  proceeds  by  setting 
Tq  •  En  ,  and,  given  T^  as  the  intersection  of  En  and  a  finite  set  of  closed 
half-spaces  containing  S  ,  picking  x^  to  maximize  f  over  T^  M  T  ,  stopping 
with  x^  optimal  if  x^  c  S  ,  and  otherwise  letting  S^  be  the  intersection  of 
En  and  a  subset  of  the  half-spaces  determining  T^  such  that  y^  maximizes  f 
over  il  T  ,**  finding  a  certain  closed  half-space  containing  S  but  not 

x^  ,  setting  T^+j  -  i'l  ,  and  continuing. 

It  was  shown  by  the  author  in  [4]  that  if  is  picked  in  a  certain  manner 

as  below  and  f  is  uniformly  concave  on  T  then  {x^}  converges  to  the  optimum. 
Eaves  and  Zangwill  [1]  allowed  the  cuts  to  be  certain  closed  convex  sets  (rather 
than  just  closed  half-spaces)  and  only  required  (essentially)  that  f  be  quasi¬ 
concave  on  T  and  strictly  quasi-concave  on  any  convex  subset  of  T  -  S  in 
proving  the  optimality  of  this  procedure  by  using  the  notion  of  a  separator.  In 
Theorem  2  the  result  of  Eaves  and  Zangwill  (with  the  cut:,  here  limited  to  certain 
closed  half-spaces  but  with  their  weakened  conditions  on  f)  is  established  by  a 
much  simpler  proof  using  the  framework  of  [4],  Theorem  3  generalizes  and  extends 
the  convergence  rate  established  in  [4], 

A  mapping  (a(x),b(x))  from  T  -  S  into  En+1  with  a(x)  e  En  and 
b(x)  e  E1  is  a  limiting  cutting-plane  function  if  S  H(x)  5  {y  :  a(x)-y  >  b(x)} 


This  compactness  assumption  is  relaxed  in  [4], 
ft 

If  f  is  pseudo-concave  on  T  then  would  satisfy  these  conditions  if  the 

constraints  dropped  are  any  of  those  constraints  determining  T,  which  are  inactive 

Iv 

at  x.  . 
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for  all  x  e  T  '  S  ,  (a(x),b(x))  is  bounded  on  T  *  S  ,  and  for  any 

{x.  :  k  ■  1,2,  ...}  Cl  '  S  with  lim  x.  -  x  e  T  *  S  the  limit  point  (a,b) 

*  k-«o 

of  any  convergent  subsequence  of  { (a(x^) ,b(x^))}  satisfies  a*x  <  b  .  A  generalized 
version  of  this  notion  was  introduced  by  Zangwill  [5],  and  examples  are  given  in 
[4]  and  [5]. 

The  following  was  proven  in  [4). 

Theorem  1: 

If  H(x)  is  determined  by  a  limiting  cutting-plane  function,  -  H(xk>  * 
and  lim  x.  ■  lim  x,  .  *  x  ,  then  x  is  optimal. 

i-~  *1  i-«o  ki+1 

Theorem  2: 

Suppose  that  H(x)  is  determined  by  a  limiting  cutting-plane  function, 

■  H(xk)  ,  and  f  is  quasi-concave  on  T  and  strictly  quasi-ccncave  on  any 
convex  subset  of  T  '  S  .  Then  the  limit  point  of  any  convergent  subsequence  of 
{x^}  is  optimal. 


Proof: 


Let  x  be  the  limit  point  of  any  convergent  subsequence  of  {x,  }  .  Since 


{x^}  is  bounded  there  then  exists  a  subsequence  ix^  l  such  that  lim  x^  =  x 

l  i)  i-*-°°  i 

and  lim  x  .  ■  y  .  Now  suppose  x  is  not  optimal.  If  x  were  feasible  it 

♦  1  A 

•*<»  i 

would  be  optimal  [4]  so  x  t  S  ,  and  by  Theorem  1,  x  +  y  .  Since  x,  maximizes 

i 

f  over  the  convex  set  S^  H  T  and  x^  e  T^  M  T  C  0  T  for  all  i  , 


>  f/ax.  +  (1  -  a)x. 


E(\)  -  f(“\ 


1+i) 


for  all  i  and  all  a  e  [0,1]  .  By  continuity, 


(1) 


f(x)  >_  f(ox  +  (1  -  a)y) 


for  all  a  c  [0,1]  . 


Since  f(x^)  is  nonincreasing  in  k  it  is  easily  seen  that  f(x)  ■  f(y)  so  by 
quasi-concavi ty 


* 


J 
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(2)  f(ax  +  (1  -  a)y)  >_  min  (f(x),f(y)}  *  f(x)  for  all  a  e  [0,1]  . 
By  (1)  and  (2), 

(3)  f(x)  ■  f(ax  +  (1  -  a)y)  for  all  a  z  [0,1]  . 

Since  S  is  closed  and  x  t  S  ,  there  exists  y  z  [0,1)  such  that 

ax  +  (1  -  a)y  t  S  for  all  a  z  [y,l]  ♦  But  by  (3)  and  the  strict  quasi-concavity 

of  f  on  the  line  segment  joining  yx  +  (1  -  y)y  and  x  , 

(4)  f(ax  +  (1  -  a)y)  >  min  {f(x),f(yx  +  (1  -  y)y)}  ■  f (x)  for  all  a  e  (y,l) 

But  (4)  contradicts  (3),  so  x  must  be  optimal.  || 

Levitin  and  Polyak  [3]  have  established  an  arithmetic  convergence  rate  for  a 
cutting-plane  algorithm  which,  when  specialized  to  subsets  of  En  ,  has 
(although  their  proof  still  holds  if  inactive  constraints  were  dropped  after  each 

subproblem)  and  uses  the  cutting-plane  method  of  Lemma  3  of  [4]  (i.e.,  a(x)  -  1 

always,  below).  Here  their  algorithm  and  method  of  proof  are  generalized  to  show 
the  same  convergence  rate  for  algorithms  which  allow  inactive  constraints  to  be 
dropped  after  each  subproblem  and  for  which  the  cutting  plane  at  x  e  T  ~  S  may 
be  generated  at  some  point  w(x)  e  1  -  S  other  than  x  on  the  line  segment 
joining  x  to  a  certain  interior  point  of  S  (as  in  Lemma  4  of  [4]).  The 
following  eliminates  the  restriction  of  the  generalization  given  in  [4]  that 
G(w(x))  <_  eG(x)  for  some  e  e  (0,1]  and  the  dependence  of  the  convergence  rate  on 
e  . 

Theorem  3; 

Suppose  that  S  =  {x  :  G(x)  >_  0  ,  x  e  T}  ,  G(x)  is  concave  and  continuous  on 

T  ,  there  exists  t  z  S  with  G(t)  >  0  ,  and  for  x  e  T  -  S  define 

A(x)  *  sup  {A  :  Ax  +  (1  -  A)t  z  S}  and  set  w(x)  *  a(x)x  +  (1  -  a(x))t  for  any 


a(x)  z  [A  (x) , 1 ]  .  Suppose  also  that  there  exists  a  function  p(w)  from  T  -  S 
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Since  t  e  S  , 

(6)  G(t)  1  G(wk)  +  uk*(t  -  *k)  . 

But  it  is  easily  seen  that  xk  i  Hk  so  xfc  +  xk+1  e  Tk+1  0  T  ■  0  Sk  fl  T 

and  by  the  strict  concavity  of  f  on  T  , 

(7)  0  ■  c<V  +  »k'<Vi  -  "k> 


or 

(8)  0  -  G(vk>  +  Vk,(Vl  -  y  +  (1  -  y\-(xk+1  -  t)  . 

By  (6)  and  (7) 

(9)  -G(t)  1  V<xk+i  '  C) 


so  by  (8) ,  (9) ,  and  (5) 


0  iG(wk)  +  yk-(kkH 


4-i) 


G(t)  +  akK| 


-  xk)  -  (1  -  ak)G(t) 

Vi  -  XJ  •  a  •  \)c(t> 


-  -  x  (i  -  y=u>  +  l*k+1  ■  y 

k 

--“k(1  -  xk)0(t)  +  akKlxk+l  •  XJ 


or 


(10) 


1  - 


A 


K 

k  -G(t) 


By  the  concavity  of  f  and  the  optimality  of  x  on  S  , 


f(xt) 


f(x)  i  f(xk)  ■  f(XitXk  +  ^  ■  Xy}^ 


£(1-  Xy)(\  ~  t)-7f(Xkxk  +  (1  -  Xfc)t) 
<d-  -  tl-l^fd^  +  (1  -  Xk)t) 


<  (1  -  xk)bd  . 


Combining  (10)  and  (11) , 


f(x.  )  -  f(x)  < 


G(t)  'Xk+1  "  Xk‘  * 


Since  Xj^x^^  e  Sk  H  T  and  Xk  maximizes  the  strongly  concave  function 


f  over  the  convex  set  fl  T  ,  for  some  y  >  0 


(13)  f(xj  >  f(ii(xk  +  xk+1))  i15f(xk)  +  4f(xu+1)  +  v|xk  -  xv+1 1 


k  k+1' 


and  from  (13) 


f(xv)  -  f(xtJ^1)  i  2y|xr  -  xk+1| 


Now  let  Dk  -  f(xk)  -  f(x)  >  0  .  By  (12)  and  (14) 


.2  „  / bdK  V 
°k  -  \G(t) / 


lXk+l  '  V 


<\  -  W 


Vi  i  Dk  -  aiDk  ■  Dk(1  -  aiV  • 


The  arithmetic  convergence  rate  for  D.  then  follows  from  (15)  as  in  [2] 


by  observing  that 


and  using  Induction  on  (16)  to  get 

or 

(17) 

As  in  (13)  and  (14) ,  it  follows  that 

(18)  Dk  -  f(xk)  -  f(x)  >_  2y|xk  -  x 
and  by  (17)  and  (18) 


pT if  +  “1“ 

k  o 


1 


k  —  1  .  -  a.lc  * 

r  +  a  k  x 

n 


lxk  "  x| 


/2ya^k 


l  r— i  g 
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